Traditionally, users have interacted with electronic devices (such as a computer or a television) or computing applications (such as computer games, multimedia applications, or office applications) via indirect input devices, including, for example, keyboards, joysticks, or remote controllers. The user manipulates the input devices to perform a particular operation, such as selecting a specific entry from a menu of operations. Modern input devices, however, include multiple buttons, often in a complex configuration, to facilitate communication of user commands to the electronic devices or computing applications; correct operation of these input devices is often challenging to the user. Additionally, actions performed on an input device generally do not correspond in any intuitive sense to the resulting changes on, for example, a screen display controlled by the device. Input devices can also be lost, and the frequent experience of searching for misplaced devices has become a frustrating staple of modern life.
Touch screens implemented directly on user-controlled devices have obviated the need for separate input devices. A touch screen detects the presence and location of a “touch” performed by a user's finger or other object on the display screen, enabling the user to enter a desired input by simply touching the proper area of a screen. While suitable for small display devices such as tablets and wireless phones, touch screens are impractical for large entertainment devices that the user views from a distance. Particularly for games implemented on such devices, electronics manufacturers have developed systems that detect a user's movements or gestures and cause the display to respond in a narrow context. For example, a user near a TV may perform a sliding hand gesture, which is detected by the gesture-recognition system, in response to the detected gesture, the TV may activate and display a control panel on the screen, allowing the user to make selections thereon using subsequent gestures, for example, the user may move her hand in an “up” or “down” direction, which, again, is detected and interpreted to facilitate channel selection.
While these systems have generated substantial consumer excitement and may ultimately supplant conventional control modalities that require physical contact between the user and a control element, current devices suffer from low detection sensitivity. The user is required to perform broad, often exaggerated and sometimes awkward movements in order to elicit a response from the gesture-recognition system. Small gestures are either undetectable or treated like noise due to the low resolution. For example, to move a cursor on the TV by a distance of one centimeter, the user's hand may have to traverse a much larger distance. This mismatch not only imposes a cumbersome operational burden on the user—particularly when there are movement constraints—but, once again, degrades the intuitive relationship between gesture and response. Moreover, the response of the system is often unitary, i.e., the physical span of a gesture always corresponds to the same on-screen control increment, regardless of user preference.
Consequently, an opportunity arises to introduce a new gesture-recognition system that detects small gestures in real time and allows users to adjust the relationship between physical movements and the corresponding actions displayed on the screen.
To select a desired virtual object displayed on the screen of the electronic device, the user may be required to sweep her hand over a large distance. Sweeping over too short a distance may be either undetectable or treated as noise due to low sensitivity, thereby causing the desired virtual object to remain unselected. As a result, the user may find herself performing the same hand gesture repetitively, with various degrees of movement, until the desired selection is acknowledged. The repetitive performance of a gesture is not only annoying but makes it difficult for the user to determine exactly when the virtual object is successfully selected. Accordingly, there is a need for a gesture-recognition system that indicates completion of the user's gesture.
Furthermore, a user action intended as a single gesture may nonetheless involve interrelated movements that can each be interpreted as a separate gesture. As a result, a conventional gesture-recognition system may not properly interpret the user's intent, and therefore convey faulty signals (or no signal at all) to a controlled electronic device. Suppose, for example, that the user waves her arm while unconsciously flexing her fingers; the gesture-recognition system may be unable to discern the intended gesture due to the interrelated movement, or may signal performance of two gestures (which may conflict, overwhelm the controlled device, or one of the gestures may fail to correspond to allowable input).
Existing systems, however, rely on input elements (e.g., computer mice and keyboards) to supplement any gesture-recognition they may perform. These systems lack the user-interface elements required for anything more than simple commands, and often, recognize these commands only after the user has set up a gesture-recognition environment via a keyboard and mouse. Therefore, a further opportunity arises to introduce a new gesture-recognition system that allows users to interact with a wider variety of applications and games in a more sophisticated manner.